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SUMMARY 


OBJECTIVE 

Implement  an  artificial  neural  network  controller  for  a  manipulator  moving  in  two-dimensional 
space.  The  motion  was  limited  to  the  shoulder  and  elbow  joints.  Two  tasks  were  defined  within  this 
effort  as  follows:  (1)  teach  the  manipulator  to  move  to  a  fixed  target  position  from  any  arbitrary  initial 
*  position,  and  (2)  teach  the  manipulator  to  move  to  any  arbitrary  target  position  from  any  arbitrary 

initial  position. 

i 

RESULTS 

Successfully  demonstrated  the  ability  to  implement  a  neural  network  on  a  manipulator  constrained 
to  planar  motion.  Specifically,  the  backpropagation  network  was  trained  to  generalize  the  manipulator 
workspace  based  on  a  limited  number  of  examples.  It  correctly  generated  the  appropriate  local  posi¬ 
tion  control  commands  to  move  the  arm  to  an  arbitrary  target. 


RECOMMENDATIONS 

As  a  result  of  this  work,  two  important  issues  have  become  apparent.  Follow-on  research  should 
address  these  issues: 

1.  The  first  issue  became  apparent  during  hardware  testing  for  the  arbitrary  target  case. 
Gravity  has  a  nonlinear  effect  on  the  required  torque  to  move  the  system  and  is  depend¬ 
ent  on  the  position  of  the  two  joints;  apparent  when  the  effective  angle  of  the  end  effector 
was  outside  the  range  of  -45  degrees  and  -135  degrees.  This  effect  would  be  eliminated  if 
a  neural  network  were  trained  to  learn  the  dynamics  of  the  arm. 

2.  The  second  issue  is  the  effect  of  a  neural  network  on  realtime  control.  The  sampling  rate 
of  the  system  drops  dramatically  after  inclusion  of  the  neural  network  controller.  Test 
results  show  the  need  to  implement  parallel  processing  hardware  as  the  complexity  of  the 
system  and  the  number  of  degrees  of  freedom  increases. 
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INTRODUCTION 


The  Ocean  Engineering  Division,  Code  94,  at  the  Naval  Ocean  Systems  Center  (NOSC)  has  con¬ 
sistently  contributed  to  the  development  of  undersea  vehicles.  As  the  requirement  for  autonomous 
underwater  vehicles  (AUVs)  continues  to  unfold,  new  and  innovative  technologies  will  be  necessary  to 
make  this  long-range  requirement  a  reality.  The  future  of  AUVs  will  be  dependent  upon  the  vehicle’s 
ability  to  perform  manipulative  tasks,  in  realtime,  in  an  unstructured  underwater  environment.  Because 
of  these  requirements,  control  algorithms  that  are  adaptable  and  self-learning  are  needed. 

Neural  networks  are  being  investigated  to  address  the  needs  of  autonomous  work  systems  required 
to  operate  in  realtime.  For  example,  figure  1  shows  a  generic  mission  scenario  for  an  AUV.  As 
shown,  the  mission  can  be  divided  into  three  basic  subtasks:  (1)  signal  processing  for  detection; 

(2)  image  processing  for  recognition;  and  (3)  robotics  for  reaction/interaction.  First,  the  vehicle  acts 
as  a  search  system,  scanning  the  ocean  floor  for  a  specified  object,  in  this  case  a  glass.  Once  the  vehi¬ 
cle  has  detected  an  object  that  has  the  qualities  of  a  glass,  the  vehicle  moves  in  for  a  closer  look.  In 
this  stage,  the  vehicle  uses  an  imaging  system  to  verify  that  the  object  is  a  glass  and  further  classify  it 
as  a  particular  kind  of  glass.  Finally,  based  on  the  object  classification,  the  vehicle  picks  up  the  glass 
and  returns  with  it.  Each  of  these  subtasks  (signal  processing,  image  processing,  and  robotics)  is  being 
addressed  using  neural  network  technology.  The  merging  of  all  of  these  disciplines  could  ultimately 
provide  an  autonomous  work  system.  The  focus  of  the  work  covered  in  this  and  future  reports  will 
cover  the  last  subtask,  that  of  underwater  manipulation. 

In  1979,  the  Undersea  AI  and  Robotics  Branch,  NOSC  Code  943,  developed  an  advanced  under¬ 
water  manipulator  to  demonstrate  new  concepts  in  supervisory  control.  This  arm  continues  to  be  an 
excellent  testbed  for  emerging  control  technologies.  It  is  an  oil-filled,  pressure-compensated,  dc-direct 
drive  arm  with  5  degrees-of-freedom.  High  torques  are  achieved  using  harmonic  drive  gearing,  and 
position  feedback  is  obtained  with  potentiometers.  The  effort  discussed  in  this  report  is  the  develop¬ 
ment  and  demonstration  of  a  neural  network  controller  on  this  undersea  testbed  manipulator. 

The  long-term  objective  of  this  work  is  to  enable  the  robot  arm  to  perform  manipulative  tasks  in 
an  unstructured  ocean  environment  in  realtime  with  minimal  operator  intervention.  In  the  ocean,  sev¬ 
eral  factors  typically  make  the  operational  environment  difficult  to  characterize.  These  may  include 
unanticipated  obstacles  (e.g.,  sea  life),  unexpected  external  forces  (e.g.,  ocean  currents),  and  poor 
visibility,  which  would  limit  any  visual  input  regarding  target  location.  All  of  these  factors  make  it  vir¬ 
tually  impossible  to  predetermine  the  operational  environment  for  a  particular  work  task  at  any  given 
time.  This  is  in  contrast  to  conventional  industrial  applications,  where  the  workspace  tends  to  be 
highly  structured  and  predictable. 

In  addition  to  a  dynamic  work  environment  associated  with  the  ocean,  the  control  of  any  robot 
arm  presents  many  challenges.  The  system  dynamics  cf  a  typical  arm  are  difficult  to  model  accurately 
and  are  highly  nonlinear,  resulting  in  many  simplifying  assumptions  to  reduce  the  complexity  and 
number  of  computations  necessary  during  control.  If  the  dynamics  of  the  manipulator  should  change, 
as  would  occur  if  system  degradation  or  damage  were  to  take  place,  the  model  would  no  longer  be 
valid.  The  manipulator  would  have  to  be  reprogrammed  to  account  for  these  changes.  Reprogramming 
i.~,  undesireable  in  an  underwater  application,  where  the  AUV  may  be  submerged  for  extended  periods 
of  time. 

Neural  networks  pose  an  ideal  solution  to  these  problems  in  that  no  a  priori  knowledge  about  the 
arm  is  required.  The  sensory-motor  relationship  is  learned  rather  than  modelled,  thus  giving  rise  to  an 
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Figure  1.  Mission  scenario  for  autonomous  vehicle. 


accurate  characterization  of  the  arm.  Also,  the  neural  network’s  inherent  adaptability  allows  it  to 
change  and  accomodate  for  any  change  in,  or  perturbation  to,  the  system.  In  addition,  the  inherently 
parallel  nature  of  neural  networks  provides  the  potential  for  realtime  autonomous  control. 


BACKGROUND 


In  conventional  robot  control,  two  models  are  solved.  The  kinematic  model,  which  is  based 
strictly  on  the  geometry  of  the  arm,  determines  the  mapping  between  joint  space  and  cartesian  space. 
The  inverse  of  this  model,  which  determines  the  joint  angles  required  to  move  to  a  position  in  the 
arm  workspace,  is  of  greater  interest.  The  dynamic  model,  which  takes  into  consideration  the  physical 
characteristics  of  the  system  (e.g.,  inertia,  friction,  damping,  and  gravity  effects  of  the  motors  and 
links),  calculates  the  joint  characteristics  (acceleration,  velocity,  position)  based  on  the  applied  motor 
torques.  The  inverse  of  this  model,  again  being  the  one  of  greater  interest,  maps  the  desired  joint 
space  to  a  torque  space  for  motor  control.  This  provides  the  torques  required  to  move  the  manipula¬ 
tor  to  a  desired  point  in  the  workspace  based  on  its  current  state.  Both  of  these  models  are  required 
for  control,  yet  their  complexity  causes  them  to  be  computationally  expensive.  Neural  networks  have 
been  posed  as  a  solution  to  eliminate  these  and  other  problems  with  conventional  methods. 

The  application  of  neural  networks  to  manipulator  control  is  still  a  field  in  its  infancy.  Most  of  the 
work  in  this  area  has  taken  place  over  the  last  4  years.  A  moderate  amount  of  effort  has  addressed 
the  inverse  kinematics  problem.  A  significantly  smaller  percentage  of  researchers  have  been  working 
on  the  inverse  dynamics  problem.  Some  work  has  also  addressed  hand-eye  coordination  through  self¬ 
learning.  Table  1  summarizes  the  documented  work  as  it  relates  to  manipulator  control.  Of  the  work 
listed  in  table  1,  only  five  authors  have  demonstrated  their  technology  on  hardware.  The  remaining 
work  has  been  devoted  to  simulations  and  paper  studies.  Refer  to  the  bibliography  for  further  infor¬ 
mation  in  this  area.  As  can  be  seen  from  the  table,  there  are  some  parallels  in  the  work  and  associ¬ 
ated  approaches,  which  is  to  be  expected  in  a  field  this  new. 

The  networks  that  have  been  applied  to  manipulator  c'ntrol  include  backpropagation.  Cerebellar 
Model  Articulation  Controller  (CM AC),  Hopfield,  and  Kohonen,  as  well  as  variations  of  these 
approaches.  In  addition,  some  work  has  been  done  based  on  biological  models,  such  as  Kuperstein’s 
INFANT  and  Kawato’s  hierarchical  neural  network. 

Several  research  areas  still  need  to  be  addressed.  As  mentioned  in  the  review  article  by  Horne, 
Janshidi,  and  Vadiee  (1990),  these  areas  include  the  following:  (1)  specifying  the  network  configura¬ 
tion;  (2)  specifying  the  teacher;  (3)  specifying  the  training  set;  and  (4)  specifying  the  learning  algo¬ 
rithm.  In  other  words,  the  field  of  neural  networks  in  robotics  still  needs  to  answer  some  fundamental 
questions.  In  addition,  to  achieve  realtime  capabilities,  the  neural  networks  must  be  readily  ported  to 
parallel  processing  hardware.  This  is  an  issue  that  has  not  yet  been  addressed,  and  yet  is  a  key  issue 
in  that  the  primary  advantage  of  neural  networks  is  their  ability  to  capitalize  on  the  concept  of  parallel 
processing.  This  will  be  especially  true  if  on-line  learning  is  expected  to  take  place,  as  is  required  if  a 
system  is  to  be  truly  adaptive. 
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Table  1.  Robotics  applications  in  neural  networks. 


Author 

Company /Institution 

Network  Used 

Approach /Application 

Barhen, 

Gulati,  Zak 

Jet  Propulsion  Laboratory 

Concept  of  Terminal 
Attractors 

Simulation  using  constrained  configura¬ 
tion  of  PUMA  560  arm  for  3-DOF  and 
7-DOF  redundant  manipulators  to  cal¬ 
culate  the  inverse  kinematic  solution. 

Bassi,  Bekey 

University  of  Southern  Cali¬ 
fornia  (USC)  Computer  Sci¬ 
ence  Department 

Backpropagation 

Simulation  on  2-DOF  manipulator 
model.  Compute  inverse  dynamics  using 
backpropagation  and  functional  decom¬ 
position  (decompose  dynamic  control 
into  three  simpler  components). 

Chen,  Pao 

Case  Western  University 

Inverse  Transfer 

Learning  scheme 

Discuss  application  to  3-DOF  arm.  No 
simulation  or  implementation  on  hard¬ 
ware.  Claim  increase  convergence  over 
similar  approach  by  Psaltis. 

Daunicht 

University  of  Dusseldorf 

N/A 

Summary  paper  of  neural  networks  in 
robotics. 

Eckmiller 

University  of  Dusseldorf 

Neural  Triangular 

Lattice 

Paper  study  on  this  neural  network 
model  (space-time  functions  represent¬ 
ing  trajectories)  for  internal  representa¬ 
tion  and  generation  of  2-D  movement 
trajectories.  Analysis  of  “draw  what 
you  see"  task. 

Graf,  LaLondc 

Carleton  University, 

Canada 

Adaptive  Arm 

Controller  (AAC) 
(variation  of  self¬ 
organizing  map  by 
Kohonen) 

Simulation  of  results  with  2-DOF  arm 
to  demonstrate  kinematic  control  for 
hand-eye  coordination.  Layers  consist 
of  obstacle  map.  arm  configuration 
map,  and  eye  configuration  map. 

Guez 

Drexel  University 

Unsupervised  Learning 
(uses  some  a  priori 
knowledge  about  arm) 

(used  backpropagation 
in  previous  work  for 
inverse  kinematics) 

Network  tested  using  a  simulated 

2-DOF  planar  manipulator.  Dynamics 
represented  by  weighted  linear  combina¬ 
tion  of  3-layer  feedforward  network 
modules.  Components  of  the  torque 
equation  are  learned. 

Guo 

University  of  Minnesota 

Hopfield 

Simulation  to  solve  the  inverse  kinemat¬ 
ic  problem  for  a  4-DOF  planar  manipu¬ 
lator.  Minimize  energy  function  defined 
as  least-squares  error  (LSE)  between 
actual  and  desired  velocity.  (Approach 
is  based  on  Jacobian  control  tech¬ 
nique. ) 

Handelman 

Princeton 

CMAC  with  rule-based 
system  as  teacher  and 
monitor 

Simulation  of  a  2-DOF  planar  arm. 
Technique:  (1)  develop  knowledge- 
based  system  components  to  accomplish 
given  control  objective;  (2)  teach  neural 
network  by  having  it  observe  and  gener¬ 
alize;  (3)  optimize  neural  network 
through  reinforcement. 

Hashimoto 

University  of  Tokyo 

Backpropagation 

Computer  simulation  of  6-DOF  arm 
with  camera  mounted  at  cnd-effector. 
Network  generates  a  change  in  joint 
angles  such  that  the  coordinates  of  vis¬ 
ual  ques.  of  image  coincide  with  desired 
ques. 

Horne 

University  of  New  Mexico 

N/A 

Summary  paper  of  neural  networks  in 
robotics. 

Table  1.  Robotics  applications  in  neural  networks  (continued). 


Author 

Company /Institution 

Network  Used 

Approach/ Application 

Josin 

Neural  Systems  Inc., 

Canada 

Modified  Backpropaga- 
tion 

Simulation  of  a  2-DOF  planar  arm. 

Uses  network  to  develop  an  inverse 
kinematic  model  which  calculates  the 
error  between  desired  and  actual  joint 
angles. 

Kawato, 

Miyamoto 

Osaka  University 

Hierarchical  neural 
network  model 

Calculate  inverse  dynamics  using  feed¬ 
back  error  learning  (uses  feedback 
torques  to  generate  the  error  signal  for 
heterosynaptic  learning).  Hardware 
demonstration  on  a  3-DOF  PUMA  260 
manipulator. 

Lee,  Kil 

use 

Bidirectional  Mapping 
Neural  Network 

Simulation  of  a  6-DOF  planar  (redun¬ 
dant)  manipulator.  Network  is  a  back- 
propagation  network  with  sinusoidal 
hidden  units.  A  feedback  loop  is  con¬ 
nected  to  the  input  to  calculate  the 
inverse  kinematic  solution. 

Kuperstein 

Neurogen 

INFANT 

Neural  controller  designed  to  learn  self- 
consistency  between  sensory  and  motor 
signals.  Inputs  are  camera  images. 

Outputs  are  arm  signals.  Teaches  a 

5-DOF  arm  to  learn  from  its  own  expe¬ 
rience  guided  by  visual  information 
from  stereo  camera. 

Martinetz, 

Ritter, 

Schulten 

University  of  Munich 

Kohonen 

Simulated  robot  arm  learns  to  position 
its  arm  and  gripper  by  observing  its  own 
movements  (inverse  kinematic  map¬ 
ping)- 

Mel 

University  of  Illinois 

Sigma-Pi 

Neurons  act  as  an  interpolating  look-up 
table.  Learns  by  randomly  moving 
through  workspace.  Control  2-DOF  arm 
planar  manipulator  with  visual  input. 

Miller 

University  of  New 

Hampshire 

CMAC 

Uses  CMAC  network  to  map  desired 
object  velocities  (image  space)  to 
actuator  drive  voltages.  Second  CMAC 
network  used  for  image  feedback  pre¬ 
dictions  transforming  drive  voltages  to 
image  velocities. 

Nguyen 

Concordia  University 

Backpropagation, 
Backpropagation  with 
error  splitting,  Func¬ 
tional  Link,  and  Coun¬ 
terpropagation 

Simulation  of  results  on  2-DOF  planar 
manipulator.  Comparison  of  networks 
for  learning  the  forward  kinematics  of 
the  arm. 

Pao,  Sobajic 

Case  Western  University 

Backpropagation 

Network  calculates  inverse  kinematic  by 
mapping  current  and  desired  position  to 
change  in  position.  Demonstrate  on 

2-DOF  planar  manipulator. 

Saxon 

Texas  A&M 

Kohonen 

Simulation  of  a  2-DOF  arm  which  gen¬ 
erates  a  mapping  between  the  work¬ 
space  (x,y  coordinate  from  camera) 
and  the  configuration  space  (angles  of 
joints  to  achieve  that  position). 

Wcrntgcs 

University  of  Dusscldorf 

Backpropagation 

Simulation  of  2-DOF  planar  arm.  Net¬ 
work  trained  to  learn  inverse  kinematics 
using  a  critic-to-teachcr  interface. 
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Table  1.  Robotics  applications  in  neural  networks  (continued). 


Author 

Company /Institution 

Network  Used 

Approach /Application 

Wilhelmsen 

University  of  Utah 

Backpropagation, 

Simulation  of  controlling  a  1-DOF  arm 

recurrence,  error- 

rotating  in  a  vertical  plane.  Network 

anticipation 

learns  inverse  dynamics  of  system. 

Yeung 

use 

Context  sensitive  learn- 

Simulation  of  the  network  learning  the 

ing  network  (uses  cas¬ 
caded  backpropagation 
networks) 

inverse  Jacobian  of  PUMA  560  arm. 

APPROACH 


To  reach  the  long-term  goal  of  achieving  autonomous  manipulative  control  in  an  unstructured 
environment,  several  simpler  steps  will  be  taken.  These  steps  are  summarized  in  figure  2.  The  focus  of 
the  FY  90  effort  was  to  develop  a  neural  network  for  controlling  a  2-degree-of-freedom  (2-DOF) 
manipulator  in  two-dimensional  (2-D)  space  with  no  visual  input,  as  shown  in  figure  2b.  This  year’s 
tasks  were  to  teach  the  network  to  (1)  start  from  any  initial  position  and  move  towards  a  fixed  target 
position,  and  (2)  start  from  an  arbitrary  initial  position  and  move  to  an  arbitrary  target  position. 

The  hardware  and  software  used  in  this  research  included  the  following: 

•  IBM-compatible  386  machine,  20-MHz  clock  speed,  with  387  math  coprocessor  chip 

•  Action  Instruments  16-channel  multifunction  input/output  (I/O)  card  (used  for  analog-to- 
digital  (A/D)  conversion) 

•  Action  Instruments  digital-to-analog  (D/A)  card 

•  5-DOF  undersea  manipulator  arm  (slave) 

•  Undersea  controller  electronics  bottle 

•  Power  supply  (provides  28  volts  dc  to  motors) 

•  Master  arm 

Figure  3  shows  the  laboratory  setup.  The  386  workstation  with  math  coprocessor  was  used  for  all  com¬ 
munication,  processing,  training,  and  control.  All  control  software  was  written  in  C.  The  neural  net¬ 
work  controller  for  the  arm  was  developed  using  Olmsted  &  Watkins  (OWL)  software,  which  was  cho¬ 
sen  for  its  easy  integration  with  existing  software.  Figure  4  shows  a  block  diagram  of  all  hardware  and 
associated  interconnections.  A  specification  of  the  manipulator  is  provided  in  appendix  A. 

The  approach  taken  for  the  FY  90  effort  involved  training  a  network  to  learn  an  iterative  trajec¬ 
tory  model  and  generate  the  appropriate  control  signals.  A  dynamic  model  is  currently  being  added 
for  learning  the  required  voltage  output  based  on  the  correction  in  joint  angle.  A  backpropagation 
learning  algorithm  was  used  for  generating  the  appropriate  input-output  relationship. 
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J2  (SHOULDER  ELEVATION) 
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Figure  2.  Autonomous  manipulator  development. 


Figure  4.  Block  diagram  of  hardware. 


FIXED  TARGET 


The  approach  taken  for  the  first  task  (fixed  target)  was  based  on  the  work  of  Pao  and  Sobajic 
(1987).  Figure  5  shows  the  coordinate  system.  The  backpropagation  network  structure,  shown  in  fig¬ 
ure  6,  consists  of  an  input  layer,  two  hidden  layers,  and  an  output  layer.  The  inputs  were  the  target 
position  designated  in  polar  coordinates  (r„  0X)  and  the  current  joint  angles  ( fi\,02 ).  The  output  from 
the  network  was  a  local  action  applied  to  each  joint  based  on  these  input  values  and  equated  to  a 
clockwise  or  counter-clockwise  motion  (+1,  -1),  or  no  motion  (0).  The  training  data  for  this  network 
is  provided  in  appendix  B,  along  with  the  resultant  weights  and  thresholds  of  the  trained  network. 
Twenty-nine  training  patterns  were  presented  during  learning.  The  desired  result  from  the  trained  net¬ 
work  was  a  generalization  of  the  workspace  based  on  this  small  set  of  data  points. 

The  procedure  for  the  fixed  target  took  place  in  three  stages.  First,  the  network  was  trained  off¬ 
line  using  the  29  data  sets  until  a  satisfactory  sum  square  error  was  reached.  Second,  this  network  was 
tested  using  a  simulation  program  to  verify  that  a  smooth  trajectory  was  generated  from  the  initial 
position  to  the  final  position.  Third,  the  network  was  implemented  on  the  manipulator  hardware. 


ARBITRARY  TARGET 

For  task  2  (arbitrary  target),  several  changes  took  place  in  the  training  data  and  the  structure  of 
the  network.  For  this  application,  a  set  of  relative  coordinates  was  chosen  to  represent  the  workspace. 
The  outputs  from  the  network  remained  the  same  as  for  the  first  task,  and  again  represented  the  local 
change  in  joint  angle.  Figure  7  shows  the  new  coordinate  system.  Figure  8  shows  the  resultant  back- 
propagation  network  consisting  of  an  input  layer,  a  single  hidden  layer,  and  an  output  layer.  The 
inputs  tc  'he  network  were  the  angle  of  the  elbow  joint  and  the  relative  angle  and  distance  between 
the  end-effector  and  the  target.  Based  on  these  three  inputs,  a  unique  combination  of  control  actions 
for  joints  1  and  2  could  be  represented  for  the  training  data.  A  total  of  142  training  sets  were  used 
for  this  application.  These  data  are  provided  in  appendix  C,  along  with  the  resultant  weights  and 
thresholds  of  the  trained  network.  The  desired  result  from  the  trained  network  was  a  generalization  of 
the  relative  workspace  based  on  a  small  set  of  data  points. 

The  procedure  for  implementation  of  this  network  was  similar  to  that  for  the  fixed  target.  The 
only  difference  was  that  some  preprocessing  took  place  before  the  data  were  input  to  the  network. 
This  preprocessing  provided  the  relative  coordinate  information  needed  by  the  network.  Again,  the 
three  stages  were  off-line  training,  testing  on  the  simulation  software,  and  implementation  on  the 
manipulator. 


RESULTS 


FIXED  TARGET 

After  a  trial-and-error  period  to  determine  the  optimum  learning  rate  (Ir)  and  momentum  (m) 
terms  for  convergence,  the  final  parameters  were  chosen  as  Ir  =  0.05  and  m  =  0.9.  The  network  was 
then  presented  the  data  repeatedly,  a  single  iteration  being  all  29  sets.  After  5000  iterations  (15  min¬ 
utes),  the  network  had  learned  a  complete  correlation  between  input  and  output  space.  As  per  the 
stntcture  of  the  OWL  software,  the  neural  network  was  then  saved.  The  network  could  later  be 
accessed  using  a  library  of  function  calls  for  both  simulation  and  hardware  control. 


Figure  5.  Manipulator  model  for  fixed  target. 
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Figure  6.  Neural  network  configuration  for  fixed  target  model 


Figure  7.  Manipulator  model  for  arbitrary  target. 


9  2  ®rel  drel 


A0,  &QZ 

Figure  8.  Neural  network  configuration  for  arbitrary  target  model. 
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The  resultant  network  was  tested  with  the  manipulator  graphics  program,  and  it  was  shown  that 
the  arm  had  learned  to  generalize  the  workspace  based  on  the  initial  29  patterns.  The  trajectories 
were  always  smooth  as  the  arm  moved  from  the  initial  position  to  the  target.  In  fact,  if  a  starting  posi¬ 
tion  was  presented  outside  the  lower  half  plane,  where  all  training  data  existed,  a  smooth  trajectory 
was  still  generated  and  the  arm  moved  directly  to  the  target.  The  final  position  of  the  arm  was  always 
within  0.5  inch  of  the  target,  which  equates  to  2  percent  of  the  total  arm  length. 

Figure  9  shows  the  neural  network  as  it  is  implemented  within  the  manipulator  hardware.  The  cur¬ 
rent  position  of  the  arm  is  determined  by  the  potentiometer  values  for  the  shoulder  and  elbow  joints. 
The  desired  target  position  is  specified  in  polar  coordinates  and  input  from  the  keyboard.  These  four 
input  values  are  presented  to  the  trained  backpropagation  network,  and  the  corresponding  local 
actions  are  determined  for  output.  The  new  desired  joint  positions  are  updated  based  on  the  following 
relation: 


^newj  =  f^old,  +  k  A 0\  . 

Therefore,  each  time  through  the  loop,  the  desired  position  is  updated  based  on  the  current  posi¬ 
tion,  given  by  the  potentiometer  values,  and  a  change  in  joint  angle  theta  is  output  from  the  network. 
These  values  are  given  to  the  manipulator  controller,  and  the  output  voltage  is  generated  for  each 
joint. 


ft  II 
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Figure  9.  Neural  network  in  relation  to  hardware  and  software. 
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ARBITRARY  TARGET 


Although  the  input-output  mapping  for  the  fixed  target  worked  well  for  the  stated  objective,  one 
limitation  made  it  unsuitable  for  the  arbitrary  target.  If  a  target  position  other  than  the  learned  posi¬ 
tion  was  input  to  the  network,  the  arm  would  still  move  to  the  learned  target  position.  The  network 
had  generalized  the  workspace  for  all  initial  positions,  but  the  learned  joint  corrections  were  based  on 
a  single  target  position. 

Building  on  the  results  from  the  fixed  target  model,  a  new  representation  of  the  data  was  pursued 
for  the  arbitrary  target.  Rather  than  provide  specific  coordinates  of  the  target  and  arm  for  training,  it 
was  more  appropriate  to  choose  relative  information  about  the  current  state  of  the  system.  The 
relative  angle  (^rel)  and  relative  distance  (drei)  between  the  end  effector  to  the  target  represented  the 
direction  the  arm  should  move  and  the  magnitude  with  which  it  should  move,  respectively.  By  specify¬ 
ing  a  third  input  parameter,  the  elbow  position,  all  ambiguity  about  the  current  state  of  the  system  is 
eliminated.  As  a  note,  in  typical  2-DOF  inverse  kinematic  problems,  there  are  two  solutions.  By  speci¬ 
fying  one  joint,  in  this  case  02,  the  arm  configuration  has  been  uniquely  defined.  Therefore,  the  net¬ 
work  receives  02,  0Te],  and  drel,  as  input. 

The  output  from  the  network  is  again  a  local  action  for  each  joint,  but  in  this  case  both  a  magni¬ 
tude  and  a  direction  were  defined  for  training.  Therefore,  large  distances  would  reflect  large  control 
actions  (+/-1),  while  small  distances  would  reflect  small  changes  (+/-0.1). 

The  training  of  this  network  was  also  different  than  the  previous  task.  Rather  than  create  one 
large  training  set  which  had  several  drC|  values,  iterations  of  smaller  data  sets  were  presented  for  learn¬ 
ing.  These  iterations  consisted  of  varying  combinations  of  02  and  0rt],  with  dre)  fixed.  The  network  was 
initially  trained  for  drc,  =  24,  then  drei  =  10,  and  finally  drcl  =  2.  The  resultant  network  commanded 
large  motions  at  great  distances  and  fine  motions  at  small  distances  from  the  target.  Each  set  of  data 
points  with  fixed  drei  was  presented  2000  times.  The  resultant  network  was  tested  on  the  simulation 
program,  and  the  arm  moved  smoothly  towards  the  target.  However,  once  the  endpoint  was  within  a 
small  tolerance  of  the  target,  it  would  begin  to  oscillate.  This  occurred  where  drc,  was  very  close  to 
zero.  The  training  data  was  therefore  increased  from  90  to  142  points,  and  the  resolution  of  about 
drei  =  0  was  improved.  This  new  training  set  was  presented  to  the  previously  trained  network  for  3000 
more  iterations.  The  results  from  this  change  in  the  data  markedly  improved  the  response  of  the  sys¬ 
tem  and  eliminated  the  oscillation  problem  around  the  target. 

Once  the  network  was  successfully  tested  in  simulation,  it  was  integrated  into  the  hardware,  as 
with  the  fixed  target.  Some  initial  difficulties  were  apparent  with  this  configuration.  The  arm  would 
move  toward  the  target  and  reach  the  final  position  if  the  shoulder  angle  was  between  -45  and  -135 
degrees.  However,  outside  of  these  limits,  where  gravity  was  resisting  rather  than  assisting  the  control 
commands,  the  arm  was  unable  to  reach  the  final  position.  The  arm  would  reach  its  static  equilibrium 
position  and  move  no  further. 

The  proportional  controller  was  no  longer  adequate  for  this  application.  The  small  steps  com¬ 
manded  by  the  network,  indicating  the  arm  was  close  to  the  target,  were  not  able  to  overcome  the 
gravitational  forces  on  the  system.  To  solve  this  problem,  a  proportional-integral-derivative  (PID)  con¬ 
troller  was  developed  to  replace  the  proportional  controller.  The  neural  network  was  then  retested  on 
the  arm  and  found  to  move  to  the  target  and  settle  approximately  1.0  inch  from  the  target  (4  percent 
of  the  arm’s  total  length).  Replacing  the  proportional  controller  with  a  PID  controller  was  only  a  tem¬ 
porary  solution  to  the  problem.  The  more  appropriate  solution  would  be  to  teach  the  network  the 
dynamics  of  the  system. 
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CONCLUSIONS  AND  RECOMMENDATIONS 


The  work  described  in  this  report  successfully  demonstrated  the  ability  to  implement  a  neural  net¬ 
work  on  a  manipulator  constrained  to  planar  motion.  Specifically,  the  backpropagation  network  was 
able  to  generalize  the  manipulator  workspace  based  on  a  limited  number  of  examples.  It  correctly 
generated  the  appropriate  local  position  control  commands  to  move  the  arm  to  an  arbitrary  target.  As 
a  result  of  this  work,  however,  some  important  issues  have  become  apparent.  It  is  recommended  that 
the  following  issues  be  pursued  during  follow-on  research. 

The  first  issue  became  apparent  during  hardware  testing  for  the  arbitrary  target.  Gravity  has  a 
nonlinear  effect  on  th^.  required  torque  to  move  the  system  and  is  dependent  on  the  position  of  the 
two  joints.  This  was  apparent  when  the  effective  angle  of  the  end  effector  was  outside  the  range  of 
-45  degrees  and  -135  degrees.  This  effect  could  be  eliminated  if  a  neural  network  were  trained  to 
learn  the  dynamics  of  the  arm.  Thus,  the  function  of  the  neural  network  in  the  control  diagram  would 
change  from  that  shown  in  figure  10a  to  the  configuration  shown  in  figure  10b. 

The  goal  of  this  work  is  to  have  a  system  that  is  adaptable  to  change.  Since  most  changes  would 
be  reflected  in  the  dynamics  of  the  arm,  this  is  a  reasonable  next  step  and  is  currently  under  investi¬ 
gation. 

The  second  issue  is  the  effect  of  a  neural  network  on  realtime  control.  Table  2  shows  that  the 
sampling  rate  of  the  system  drops  dramatically  after  inclusion  of  the  neural  network  controller.  The 
sampling  rate  drops  by  a  factor  of  60.  The  master/slave  controller,  which  performs  mostly  integer 
operations,  achieves  the  sampling  limit  of  the  A/D  card  (3000  Hz).  This  sampling  rate  drops  to  50  Hz 
when  the  neural  network  is  added.  If  some  simple  graphics  are  added  to  show  movement  of  the  arm, 
this  immediately  drops  to  5  Hz.  These  results  indicate  the  need  to  implement  parallel  processing  hard¬ 
ware  as  the  complexity  of  the  system  increases  (e.g.,  adding  more  degrees  of  freedom,  or  moving  to  a 
three-dimensional  space). 

Table  2.  Variation  in  sampling  rate  for  undersea  manipulator. 


Configuration 

Sampling  Rate 

Master/slave  only 

(no  graphics  or  neural  netwrork) 

3000  Hz 

Neural  network 

(no  graphics) 

50  Hz 

Neural  network* 

(with  graphics) 

5  Hz** 

'current  arm  configuration 

"indicates  a  need  to  move  to  parallel  processing  for  FY  91. 

Commercial  hardware  is  available  for  increasing  training  time,  but  typically  these  are  dedicated 
processing  boards  such  as  SAIC’s*  Delta  II  or  Hecht-Neilson’s  ANZA  board.  Chips  that  provide  any 
increase  in  processing  time  for  a  particular  application  are  currently  used  after  training  is  completed 
using  neural  network  software.  The  weights  resulting  from  the  trained  network  are  then  downloaded 
onto  the  neural  chip  for  operational  use.  Intel’s  ETANN  (Electronically  Trainable  Artificial  Neural 
Network)  chip  is  being  developed  to  speed  up  training  time,  but  is  still  in  the  developmental  stages. 


'Science  Applications  International  Corporation  (SAIC). 
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Figure  10b.  Proposed  dynamic  neural  network  controller. 


Processor  size  becomes  an  issue  when  considering  its  implementation  on  an  autonomous  under¬ 
sea  vehicle.  In  this  type  of  application,  large  computers  cannot  be  tolerated  due  to  volume  constraints 
of  an  underwater  vehicle.  The  large  computer  workstations  used  in  many  neural  network  applications 
are  clearly  not  a  viable  solution  to  the  realtime  problem  for  this  application.  Therefore,  implementa¬ 
tion  of  neural  network  chips  will  be  the  method  of  choice,  which  will  be  followed  as  this  technology 
unfolds. 

Both  dynamic  control  and  realtime  processing  will  be  addressed  during  FY  91  under  the  Inde¬ 
pendent  Exploratory  Development  program  at  NOSC. 
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Manipulator  assembly  drawing  showing  the  movement  angle  for  each  joint  (Bosse,  Heckman,  1973). 
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Input  width:  4 

Layers  NOT  counting  input:  3 
Connect  inputs  to  outputs:  NO 
Enable  batching  and/or  momentum:  YES 


Layer  1 :  6 

states 

30 

weights 
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Layer  2:  4 

states 

28 

weights 
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,  03691 e- 11 
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Instantiation  Parameters: 

Input  width:  3 

Layers  NOT  counting  input:  2 
Connect  inputs  to  outputs:  NO 
Enable  batching  and/or  momentun:  YES 


Layer  1:  15 

states 

60  weights 

States: 

0.128261 

0.957379 

0.0654994 

0.116211 

0.169312 

0.0102753 

0.268152 

0.842259 

0.121647 

Weights: 
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Layer  2:  2 

states 

32  weights 
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-2.85941e-07  5 
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Weights: 
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